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ABSTRACT 


Box  and  Draper  (1965)  introduced  a  criterion  for  the  estimation  of  parameters  from  a 
multiresponse  model.  This  criterion  can  lead  to  misleading  results  in  the  presence  of  linear 
relationships  among  the  responses.  Box  et  al.  (1973)  proposed  a  procedure  for  detecting  the  existence 
of  such  relationships  when  the  multiresponse  data  are  subject  to  round-off  errors.  The  procedure, 
however,  can  be  adversely  affected  by  large  differences  in  the  orders  of  magnitude  of  the  responses  as 

well  as  in  the  units  of  measurement  on  which  the  responses  are  expressed.  It  is,  therefore,  necessary  to 

-  ..  - 

scale  the  responses  prior  to  the  application  of  that  procedure.  In  this  article,  I  discuss  the  effect  of 
scaling  on  the  implementation  of  the  eigenvalue  analysis  by  Box  et  al.  (1973).  Two  numerical 
examples  are  given  for  illustration.  •  /  -' 

KEY  WORDS:  Multiresponse  model;  Box-Draper  estimation  criterion;  Round-off  error;  Linear 

dependencies}  Ill  conditioning;  Eigenvalue  analysis. 
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1.  INTRODUCTION 


Consider  the  general  multiresponse  model 


yui  =  (-u’  +  eui’  u=1,2 . n;  i=1’2’"”r’ 


yul  is  the  value  of  the  i~  response  at  the  u~  experimental  run, 

7j(xu,  0)  is  the  expected  value  of  y u* , 

xu  is  a  vector  of  values  of  k  input  variables,  denoted  by  Xj,  X2,  •••,  ,  at  the  u~ 

experimental  run, 

8  is  a  vector  of  p  unknown  parameters,  and 
is  a  random  error  associated  with  yuj. 

Let  y.  and  j j  denote,  respectively,  the  vector  of  values  of  the  i~  response  and  the  associated 
vector  of  random  errors  (i=l,2,...,r).  The  multiresponse  model  in  (1.1)  can  be  written  in  the  form 

Y=F  +  e,  (1. 

where  Y  =  [y^  :  :  yr],  F  is  an  nxr  matrix  whose  (u,i)“  element  is  ^j(xu,0)  and 

«  =  p!  :  £2  :  :  5r]- 

According  to  Box  and  Draper  (1965),  estimates  of  the  elements  of  9  can  be  obtained  by 
minimizing  the  determinant,  |r(0)|,  of  the  matrix  r(0)  with  respect  to  9,  where 

T($)  =  (Y  -  F)'(Y  -  F).  (1. 

We  refer  to  this  method  of  estimation  as  the  Box-Draper  estimation  criterion.  Box  et  al.  (1973) 
pointed  out  that  when  linear  relationships  exist  among  the  columns  of  Y,  the  matrix  r(0)  becomes 
singular,  that  is,  |r(0)|  =0.  In  practical  situations,  the  multiresponse  data  are  subject  to  round-off 
errors,  which  cause  the  determinant  of  T(f )  to  be  different  from  zero  and  to  change  as  9  is  changed. 
Minimization  of  this  determinant  under  such  conditions  will  produce  nonsensical  results  (Box  et  al. 
1973  and  McLean  et  al.  1979). 


In  order  to  detect  singularities  in  r(0),  Box  et  al.  (1973)  proposed  a  procedure  in  which  the 


eigenvalues  of  the  matrix  Diy  are  examined,  where 

P  =  Y'(In  -  Jn/n).  (14) 

In  (1.4),  In  and  Jn  denote  the  identity  matrix  and  the  matrix  of  ones,  respectively,  both  of  order  nxn. 
Box  et  al.  (1973)  showed  that  m  linearly  independent  relationships  must  exist  among  the  responses  if 
and  only  if  the  matrix  pp;  has  a  zero  eigenvalue  of  multiplicity  m.  As  mentioned  earlier,  none  of  the 
eigenvalues  of  DD/  will  be  exactly  equal  to  zero  because  of  round-off  errors  in  the  multiresponse  data. 
Small  eigenvalues  of  DDf  should,  therefore,  be  examined  to  determine  if  they  correspond  to  linear 
dependencies  among  the  responses. 

If  A'  is  a  small  eigenvalue  of  DD^  then  in  the  presence  of  round-off  error  only  and  for  a 
sufficiently  small  <5,  the  expected  value  of  A*  is  approximately  equal  to 

)  =  (n-l)<T2e,  (1-5) 

9  2 

where  <r(re  —  S  /3  is  the  round-off  error  variance.  Formula  (1.5)  is  valid  under  the  assumption  that 
the  round-off  errors  are  statistically  independent  and  have  the  uniform  distribution  U(-<5,<5).  See  Box  et 
al.  (1973).  An  approximate  upper  bound  on  the  variance  of  A’  was  given  in  Khuri  and  Conlon  (1981) 
as 

Var(A')  <  pJE  +  nr(nr-l)  -  (n-l)2J(r?e.  (1.6) 

Formulas  (1.5)  and  (1.6)  can  be  used  to  determine  whether  a  small  eigenvalue  of  PPf  should 
be  considered  as  zero,  an  indication  of  a  singularity  in  the  matrix  T((?)  in  (1.3).  Box  et  al.  (1973) 
discussed  possible  remedies  when  such  a  situation  occurs,  which  include  dropping  some  of  the  responses 
that  are  influential  contributors  to  the  singularity. 

Quite  often,  the  responses  have  different  units  of  measurement,  which  cause  them  to  have 
widely  different  orders  of  magnitude.  Furthermore,  the  round-off  errors  in  all  the  responses  mav  not  be 
identically  distributed  as  U(-6,<5).  All  of  these  factors  can  seriously  affect  the  eigenvalue  analvsis 
described  earlier  as  will  be  seen  in  more  detail  in  Section  3.  To  remedy  this  numerical  inconsistency, 
the  responses  should  be  scaled  before  proceeding  with  the  eigenvalue  analysis.  This  action  will 
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obviously  alter  formulas  ( 1.5)  and  ( 1.6)  for  the  expected  value  and  variance  of  a  small  eigenvalue  of 
PPV  A  modification  of  these  formulas  will  be  presented  in  Section  3. 


2.  SCALING  OF  THE  RESPONSES  AND  MEASURES  OF  NEAR  SINGULARITY 

In  a  general  multiresponse  situation,  the  response  variables  usually  have  distinct  physical 
meanings,  distinct  units  of  measurement,  and  widely  different  values.  Such  scale  imbalance  makes  it 
difficult  to  interpret  the  results  of  the  eigenvalue  analysis  as  described  in  Section  1.  To  avoid  this 
difficulty,  the  responses  must  be  scaled  first. 

Let  W  be  an  rxr  diagonal  matrix  whose  i~  diagonal  element,  w.,  is  defined  as 

w|  =  £  (Xui  '  ?[)2  5-  >=1-2 . r,  (2.1) 

[_u=l 

where  yui  is  the  value  of  the  i~  response  at  the  u~  experimental  run  and  y;  =  (£  y  )/n.  Consider 

u=l 

the  nxr  matrix  S  given  by 

S  =  YW'1  (2.2) 

where  Y  is  the  matrix  of  multiresponse  data  in  (1.2).  The  matrix  S  is  a  linear  transform  of  Y,  which 
results  from  dividing  the  i  column  of  Y  by  Wj  (i=l,2,...r).  It  is  known  that  estimates  of  the 
parameters  from  the  multiresponse  model  (1.2)  are  invariant  under  this  scaling  convention  (see  Bates 
and  Watts  1985,  p.  (530).  Using  (2.2)  in  (1.4)  we  obtain 

P  =  VVS'Un  -  Jn/n).  (2.3) 

Let  us  now  define  the  rxn  matrix,  B.  as 
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B  =  S'(ln  -  J„/n). 


(2-4) 


From  (2.3)  and  (2.4)  we  have 

B  =  W1D>  (2.5) 

or  equivalently, 

B7  =  (In  -  Jn/nJYW'1.  (2  6) 

We  note  that  the  columns  of  B7  have  unit  lengths  and  that  the  matrix  BB7  is  in  correlation  form. 
Furthermore, 

BB'  =  W^PP'W'1  (2.7) 

Thus,  when  the  responses  are  scaled  as  in  (2.2),  the  matrix  DD7,  used  in  the  eigenvalue  analysis  of  Box 
et  al.  (1973),  is  transformed  into  the  matrix  BB7. 

In  case  of  a  singularity  in  the  matrix  T(f )  (see  1.3)  and  in  the  presence  of  round-off  errors  in 
the  multiresponse  data,  both  DD7  and  BB7  are  near  singular,  that  is,  their  determinants  are  close  to 
zero.  In  this  case,  the  columns  of  D 7  as  well  as  those  of  B7  are  nearly  linearly  dependent,  or 
multicollinear.  In  other  words,  if  a  singularity  exists  in  the  matrix  C(0),  then  both  D7  and  B7  will 
suffer  from  ill  conditioning.  Their  degrees  of  ill  conditioning,  however,  can  be  quite  different. 

As  a  measure  of  ill  conditioning  of  the  matrix  B7  and  that  of  D7,  I  use  the  condition  numbers. 
*(B7)  and  k(D7),  respectively.  By  definition,  the  condition  number  of  B7  is 

«(B7)=[emax(BB7)/emin(BB7)]l 
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where  emax(BB/)  and  emjn ( P B , )  denote  the  largest  and  smallest  eigenvalues  of  the  matrix  BBV  The 
condition  numt-.r  of  D/  is  similarly  defined.  The  larger  the  condition  number,  the  more  ill  conditioned 
the  matrix.  Another  indicator  of  ill  conditioning  is  provided  by  the  variance  inflation  factors  (VIF). 
Since  the  columns  of  B;  are  centered  and  scaled  for  unit  length,  the  VIF’s  associated  with  this  matrix 
are,  by  definition,  the  diagonal  elements  of  the  matrix  (BB,)‘^.  Large  VIF's  indicate  ill  conditioning 
and  can  help  diagnose  its  nature.  More  specifically,  large  VIF’s  correspond  to  responses  that  are 
involved  in  the  multicollinearity  in  the  columns  of  Bf. 

It  was  mentioned  earlier  that  the  degree  of  ill  conditioning  of  D '  can  be  quite  different  from 
that  of  B?  To  see  this,  let  us  consider  (2.5)  and  the  inequality  given  in  Belsley  et  al.  (1980,  p.  182) 
Then, 

*(B')  >  k(D')/k(W).  (2.8) 

Since  W  is  diagonal,  its  condition  number  is  given  by 

«(W)  =  max(w.)/min(w.),  (2.9) 

i  1  i  i 

where  min(w.)  and  max(wj)  denote,  respectively,  the  smallest  and  largest  of  the  Wj’s  defined  in  (2.1). 
Inequality  (2.8)  can  be  rewritten  as 


(2.10) 


If  «(D/)  is  very  large  and  min(wj)  is  very  small  as  compared  to  maxfw.j,  then  can  be  small, 

but  need  not  be.  Thus,  ill  conditioning  can  be  improved  by  the  scaling  convention  of  (2.2),  especially 
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when  the  w-'s  in  (2.1)  are  widely  different.  As  a  matter  of  fact,  this  kind  of  scaling  has  “near-optimal' 
properties  in  the  sense  that 


*(?')  <  rr  min  [k(B'G)], 


(2.11) 


where  Q  denotes  the  set  of  all  nonsingtilar  diagonal  matrices  of  order  rxr  (see  Belsley  et  al.  1980,  p. 
1S5).  In  particular,  from  (2.5)  and  (2.11)  we  conclude  that 


x(B')  <  ^Tr/c(p'). 


(2.12) 


Thus,  x(B;)  cannot  be  off  by  more  than  a  factor  of  'Tr  from  the  minima]  condition  number  given  on 
the  right-hand  side  of  (2.11). 

Improving  the  conditioning  of  D*  through  the  transformation  (2.5)  is  desirable  since  a  severely 
ill-conditiored  matrix  is  sensitive  to  round-off  errors  in  its  entries  (Maron  1982,  p.  210).  It  follows  that 
the  eigenvalue  analysis  of  Box  et  al.  (1973)  can  be  safeguarded  from  the  effects  of  ill  conditioning  by 
adopting  the  scaling  convention  of  (2.2).  Of  primary  importance  in  that  analysis  is  the  magnitude  of 
the  smallest  eigenvalue  of  the  matrix  DD^,  where  D  is  given  in  (1.4).  This  can  be  quite  different  from 
the  smallest  eigenvalue  of  BB^  as  lemma  2.1  shows,  especially  when  the  w.’s  in  (2.1)  are  markedly 


different. 


1-  Let  B,  D,  and  Wj(i=l,2,...,r)  be  defined  as  in  (2.4),  (1.4), and  (2.1),  respectively.  Let 


em;n( *)  and  emax(')  denote  the  smallest  and  largest  eigenvalues  of  a  symmetric  matrix.  Then, 


min(w?)  <  -.min <  max(w?). 

i  “  «  •  (BB')  ~  i 


(2.13) 


Proof.  See  Appendix  A. 


In  (2.13),  Dp'  and  BB'  are  computed  using  rounded-off  response  values.  Consequently,  both 
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matrices  are  nonsingular  even  when  the  responses  are  linearly  dependent.  From  (2.13)  we  note  that 

e  •  (DD^  can  be  heavilv  affected  bv  extreme  values  of  w.  ( i=  1,2, — ,r ).  By  contrast,  e  ■  (BB  )  <  1 
min '  -  -  ■  •  i  “““ 

since  BB'  is  in  correlation  from  (the  determinant  of  this  matrix,  which  is  the  product  of  its 
eigenvalues,  is  less  than  or  equal  to  one  by  Theorem  8.7.6  in  Graybill  1983). 


3.  THE  EXPECTED  VALUE  AND  VARIANCE  OF  A  SMALL  EIGENVALUE  OF  BBf 
IN  THE  PRESENCE  OF  ROUND-OFF  ERRORS 

In  this  section,  the  expected  value  and  variance  of  a  small  eigenvalue  of  BB/  in  (2.7)  are 
derived  when  round-off  error  is  the  only  error  present.  The  derived  values  can  be  used  in  an  eigenvalue 
analysis  to  determine  whether  a  small  eigenvalue  of  BB'  is  in  fact  a  zero  eigenvalue  in  the  absence  of 
round-off  error.  By  (2.7),  BB;  and  DD*  are  of  the  same  rank.  Hence,  if  B^  has  a  zero  eigenvalue, 
then  so  does  DD,1  which  indicates  the  presence  of  a  linear  relationship  among  the  responses. 

Let  yuj  be  the  exact  i~  response  value  at  the  u~  experimental  run,  and  let  y^j  be  the  value  of 
y  I  rounded  off  to  a  certain  number  of  decimal  places  (u=l,2,...,n;  i=l,2,...,r).  In  the  presence  of 
round-off  errors  only  we  have  the  model 

Y*  =  Y  +  A,  (3.1) 


where  Y”  and  Y  are  the  nxr  matrices,  (yV)  and  (yuj),  respectively,  and  A  is  the  nxr  matrix  (Ayuj) 
of  rounding  errors  in  the  response  values.  Let  Aj  be  the  vector  of  round-off  errors  for  the  i—  response 
(i=l,2,...,r),  that  is. 


A;  =  (Ayl;.  Av2j .  Ayni)',  i=1.2. 


(3.2) 
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K  .  . 

- 1  ..%  ♦  it**  p.ir'i.i.  m-nx  .it iv c  i!  i  v  ,  .  s .  v  i  with  respect  to  •  t  u.v  =  1 ... 

>\  !  1  •  1  i  •  Ji  •  ni  r  •  v  i 


Let  is  -up  [*>•*•  •  hat  :n  'he  absence  of  round-off  error,  the  matrix  BB/  in  1  — . T  i  has  a  zero 
eige.uva.ue  of  mu  Tpm'itv  :n  •  >  •  >.  This  means  that  m  linearis  independent  relationships  must  exist 
intone  Tie  responses  [  et  a,  a.,.  ..  am  be  an  orthonormal  set  of  eigenvectors  of  BB^  for  the 
eigenvalue  zero.  If  Tie  rounded- off  multiresponse  data  matrix.  V’,  is  used  in  the  computation  of  B  ;n 
2  V,  Tien  'he  matrix  BB*  will  have  m  small  eigenvalues.  Let  Aj  (j=l,2 . m)  denote  the  j~  smallest 


eigenvalues  Then,  for  sufTicientlv  small  6^.  <5., . Ar  we  have 

-  a j i  AS  i,i  I n  Jn/n)(  \S)a  .  j=1.2 . m. 


(3.7, 


a  tie  re  , 


are  the  parameters  of  the  uniform  distributions  associated  with  the  round-off  errors 


trom  the  r  responses  see  Wilkinson  1963.  p.  136;  Khuri  and  Colon  1981,  p.  372).  The  symbol  ~  in 
3  7  i  means  that  the  two  sides  are  of  the  same  order  of  magnitude.  The  eigenvectors,  a^,  a9,  ...,  am, 
that  appear  in  ■  3.7)  can  be  approximated  with  a9,  ....  am,  an  orthonormal  set  of  eigenvectors  of 
BB/.  which  correspond  to  Aj.  A  A .  A^,,  respectively.  We  can  then  write 


Aj  ~  )'( l n  -  Jn/n)(AS)aj.  j=l,2 m. 


(3.8) 


1  sing  the  expression  in  (3.8).  it  can  be  shown  (see  Appendix  B)  that  we  approximately  have 


L,  0,0,  u 


E( A- J  =  (n-21  V  iT"a-./w".  j  =  l,2 
J  >  U7  ‘  J 


.m. 


(3.9) 


rhere  IT~—  f'~  3  is  ’he  rounding  error  variance  for  response  i  (i=1.2 . r). 
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Smt  ’nir' V 


•  m*C,  f  lT.  /.  «•. 


y  w  m  i'.'j 

Lbi  Ho*  Am  .  AJ*.  A-  jV.".  aJ“.  V,'  A-’ 


(3.10 


Vat  (At)  <  [y  +  n(»-l)].£ Ti"t 


i<£ 


i=l 


j=l,2,...,m, 


here  a-,  is  the  i—  element  of  a-  (i=l,2,...,r;  j=l,2,...,m)  and  r-  is  given  by 


ri  =  [(n-1)  w?  +  ny?J/w?,  i=l,2 . r. 


(3.11) 


If  Aj  is  of  the  same  order  of  magnitude  as  E(Aj),  then  we  may  treat  A:  as  a  zero  eigenvalue 
and  conclude  that  a  linear  dependency  exists  among  the  responses.  The  elements  of  the  corresponding 
eigenvector,  §j,  may  be  used  to  identify  the  linear  dependency.  The  upper  bound  in  (3.10),  denoted  by 


can  be  used  to  obtain  the  standardized  values 


”j  *  [Ai '  E<  W-Fi 


,  j— l,2,...,m. 


(3.12 


Large  values  of  r)-  clearly  indicate  that  Aj  does  not  correspond  to  a  zero  eigenvalue  of  BB*.  Small 
values  of  r/.,  however,  do  not  necessarily  imply  a  zero  eigenvalue  since  Uj  is  just  an  upper  bound  to  the 


true  variance  of  A*. 


J 


4.  NUMERICAL  EXAMPLES 

Two  examples  are  presented  in  this  section  to  illustrate  the  implementation  of  the  eigenvalue 
analysis  described  in  Section  3. 

4.1  Example  1 

Let  us  consider  the  data  on  the  thermal  isomerization  of  a-pinene  at  189.5',  which  was 
reported  in  Fuguitt  and  Hawkins  (1947)  and  analyzed  by  Box  et  al.  (1973).  The  data  are  reproduced 
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in  Table  1.  In  this  example,  values  from  the  five  responses  have  been  recorded  to  the  nearest  .10,  hence 
=  dr,  =  6 ^  =  *4  =  ''5  =  05.  where  the  iT’s  are  the  parameters  associated  with  the  uniform 
distributions  of  round-off  errors.  The  matrix  BB7  is  given  in  Table  2  along  with  its  corresponding 
variance  inflation  factors  and  the  condition  number,  k(  B ^).  The  eigenvalues  of  BB7,  their 
corresponding  expected  values,  values  of  rjj  (see  3.12),  and  the  matrix  of  orthonormal  eigenvectors  of 
BB7  are  given  in  Table  3. 

The  first  and  second  smallest  eigenvalues  of  BB’  in  Table  3  are  of  the  same  order  of  magnitude 
as  their  expected  values.  The  associated  values  of  tj.  (j=4,5)  are  very  close  to  zero.  These  two 
eigenvalues  can,  therefore,  be  regarded  as  equal  to  zero.  The  elements  of  the  corresponding 
eigenvectors  (the  last  two  columns  of  the  matrix  in  Table  3)  can  be  used  to  define  two  linearly 
independent  relationships  among  the  five  responses.  The  third  smallest  eigenvalue  of  BB7.  namely 
.0027773,  is  not  of  the  same  order  of  magnitude  as  E(Aj).  The  associated  value,  however,  is 
relatively  small.  This  eigenvalue  corresponds  to  a  linear  relationship  among  the  expected  values  of  the 
responses  (see  Box  et  al.  1973,  Section  6).  The  remaining  two  eigenvalues,  namely  .881599  and 
4.11521,  are  much  larger  and  have  large  rjj  values  (j=l,2).  Hence,  they  are  not  associated  with  any 
linear  relationships  among  the  responses  or  among  their  expected  values. 

The  large  variance  inflation  factors  and  condition  number  in  Table  2  indicate  that  the  matrix 
B7  is  severely  ill  conditioned,  hence  it  is  very  sensitive  to  round-off  errors.  We  note  that  the  responses, 
yv  y<>.  y4  and  v^  are  quite  involved  in  the  linear  dependencies  since  their  corresponding  variance 
inflation  factors  are  extremely  large.  This  is  also  supported  by  an  examination  of  the  elements  of  the 
last  two  columns  of  the  matrix  of  eigenvectors  in  Table  3.  In  both  columns,  the  third  element,  which 
corresponds  to  Vj,  is  the  smallest  in  absolute  value. 

4.2  Example  2 

Research  was  conducted  by  Ahmed  et  al.  ( 1983)  at  the  I’niversity  of  Florida  in  order  to 
develop  acceptable  fish  patties  from  sheepshead  harvested  along  the  Florida  roast.  Deboned  and 
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washed  sheepshead  flesh  was  mixed  with  varying  proportions  of  x^  =  sodium  chloride,  X2  =  sodium 
tripolyphosphate,  and  X3  =  sodium  alginate.  Of  interest  was  the  determination  of  the  effects  of  these 
three  input  variables  on  the  texture  quality  of  cooked  patties.  This  was  measured  by  the  values  of  four 
response  variables,  namely,  y^  =  breaking  force  in  grams,  y0  =  texture  firmness,  y^  =  texture 
preference,  and  y^  =  flavor.  The  last  three  responses  were  measured  using  a  nine-point  rating  scale 
with  1  being  least  desirable  and  9  most  desirable.  The  response  values  are  given  in  Table  4. 

The  variance  inflation  factors  in  this  example  are  2.374,  28.653,  24.244,  1.768,  and  the 
condition  number  of  the  matrix  B*  is  k(B')  =  12.552,  an  indication  of  moderate  ill  conditioning.  The 
values  of  tj.  in  (see  3.12)  are  =  1909.61,  =  392.03,  =  214.702,  and  rj ^  =  12.12.  Hence,  no 

eigenvalue  of  BB7  can  be  regarded  as  equal  to  zero. 

It  is  interesting  here  to  note  that  the  eigenvalues  of  the  matrix  BB/  are  .0192,  .3397,  .6202, 

and  3.0210.  By  contrast,  the  eigenvalues  of  the  matrix  DD*  for  the  unsealed  responses  are  .2871, 

1 

6.8863,  15.8812,  and  2,009,  864  with  a  condition  number,  k(D')  =  (2,009,864/.287lf2  =  2645 .86.  The 
latter  number  greatly  exceeds  *(B/).  This  shows  that  scaling  of  the  responses  can  improve  the 
conditioning  of  the  matrix  BB/,  which,  in  turn,  reduces  its  sensitivity  to  round-off  errors. 

5.  CONCLUDING  REMARKS 

The  scaling  convention  proposed  in  (2.2)  is  recommended  to  be  used  prior  to  the  application  of 
the  eigenvalue  analysis  for  detecting  linear  dependencies  among  the  responses.  Scaling  removes 
inconsistencies  in  the  units  of  measurement  by  putting  all  response  variables  on  a  common  scale.  An 
eigenvalue  analysis  applied  directly  to  the  original  data,  e.g.,  the  data  of  Example  2  in  Section  4.2,  is 
not  always  meaningful  because  the  responses  may  not  have  the  same  scale. 

As  was  observed  in  Example  2,  round-off  error  can  be  reduced  by  scaling.  To  see  this  in 
general,  we  note  that  if  the  Wj's  in  (3.9)  exceed  unity  and  if  the  <t?’s  are  equal  to  <Tre,  then  the 


o 

expected  value  of  Aj  is  less  than  (n-l)o-fe.  The  latter  quantity  is  given  in  Box  et  al.  (1973)  as  the 
expected  value  of  a  small  eigenvalue  of  DD*. 
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Appendix  A:  Proof  of  the  Double  Inequality  (2.13) 


From  (2.5)  we  have  that 


DP'  =  WBB'W, 


(A.l) 


where  W  is  the  rxr  diagonal  matrix  whose  i —  diagonal  element,  w^,  is  given  in  (2.1).  Then,  from 
(A.l)  and  Lemma  2  in  Khuri  (1986,  p.  355)  we  can  write 


"min  minfwf). 


Let  e^jn(-)  denote  the  smallest  positive  eigenvalue  of  a  symmetric  matrix.  Since  the  nonzero 
eigenvalues  of  WBB'W  are  equal  to  those  of  then 


«min(WBB'W)  =  4in(B'W2B) 

<  e^in(B,B)emax(W2)  (see  Khuri  1986,  p.  356)  (A.3) 

=  emin^')  miax<w?)- 


The  double  inequality  in  (2.13)  now  follows  from  (A.l),  (A. 2),  and  (A.3). 


Appendix  B:  The  Derivation  of  the  Expected  Value  in  (3.9)  and  the  Upper  Bound  in  (3.10) 


i)  The  Expected  Value  in  (3.9) 


From  (3.5)  we  have  that 


8u(ylit  y2j . yni)  =  yui/  E(yvi-yi)22.  u=u . n;  i=i,2 — r 

V=1 


By  taking  the  partial  derivatives  of  gu  with  respect  to  its  arguments  we  get 


dgu  _ 

dy  • 

j  VI 


(1/w. )  -  (y  *  '  y-)y  -/w- 

V  /  1  /  \J  Vj  /i/-rui'  1 


-  (y  •  -  y.  )y  •  /w.  , 


where  w.  is  defined  m  (2.1).  Let  0  .  be  the  nxl  vector 


* .  =  rl&L ,  |«u 
'U‘  ^  ^2i 


-  fe)' 


We  can  then  write  (3.6)  in  the  form 


As  ■  as  <s>'  ■  A- , 
ui  "ui  -  r 


where  Aj  is  defined  in  (3.2).  A  first-order  approximation  of  AS  =  (As^j)  in  (3.4)  can  then  be 
expressed  as 


AS  as  ^ '<t>  |A|  :  <t>  A  ^  *.  ■■■  : 


-16- 


From  (3.3)  and  (B  6)  we  get 


E(*;A-)  =  0,  i=l,2,..,r 

Var(0iAj)  =  i=l,2,...,r, 

9  2  th 

where  ar  —  6?/ 3  is  the  round-off  error  variance  for  the  i—  response  (i=l,2,...,r). 

Let  us  now  write  the  expression  in  (3.8)  as 

A-  ~  bj  (In  -  J„/n)bjt  j=l,2 . m. 


(B.7) 


where 


j— 1.2 . m, 


where  a.  —  is  the  i  element  of  ij.  From  (B.7),  (B.9),  and  the  fact  that  the  Aj’s  are  statistically 
independent  we  approximately  have 


E(bj)  =  0. 


JL  .  o  •> 


j=  1 ,2,..,m 

Var(b.)  =  £  if. a f 0.0',  j=l,2 . m. 

J  i=1  ‘J 

The  expected  value  of  Aj  in  (B.8)  can  then  be  approximately  written  as 


E(  A? )  =  tr 


(In  -  Jn/n)E 


=  L.  (r~a|j  ‘In  '  J n/n ><?i J-  j=l.- . m. 


(B.l 


Since  the  u  row  of  <j  i  u=l,2 . n;  i=l,2..  .,r)  is  of  the  form  given  by  (B.l!)  and  (  B.3).  then  it  is  easy 

to  show  that 


'!n  '  Jn/n)<?.  -  r  !n  •  b4"  ■] n  •  H-,  i— 1.2 . r 

i  i  wJ 

i 

where  H  is  a  symmetric  n*n  matrix  whose  element  is 

=  <-%,  -  •  Vjb  (*.V  -  1.2 . n;  i=l,2,.  ,.r. 

We  note  that  J n H •  =  0  for  i=l,2 . r.  It  follows  that 


(B  11 


( B.  13 


(B.14 


Formula  (3.9)  follows  from  (B.ll)  and  (B.14) 
ii)  The  Upper  Bound  in  (3.10) 


From  (3.8)  we  approximately  have 


Var(Aj)  =  E  raj(AS)'(!n  -  Jn/n))(  AS)aJ2  -  E(  Aj )  '“.  j=l.2 . m.  (B.  15 


We  note  that 


E  r4!(ASl'(In  -  Jn/n))(AS)a;12  <  E  fa[(  AS)'(  AS)a;]2, 


1=1.2 


m. 


(B.16 


A 


since  ( AS)'(AS)  -  (AS)'(I„  -  Jn/n)(AS)  is  a  positive  semidefinite  matrix.  We  also  note  that 


Efa'lASi'lASlaJ2  <  E{tr[(AS)'(A S)]}5 


(B.  IT) 


!(AS/(AS)a-  is  less  than  or  equal  to  the  largest  eigenvalue  of  (AS/(AS),  Lancaster  (1969,  p. 


since  4j(AS)  (AS )a^ 


From  (B.4)  and  (B.17)  we  approximately  have 


E  ra((AS)'(AS)fi'|2  <E  t  t  (^i^)2  2 

LJ  JJ  u=ii=i 


<z\t  i  (*ii*oi><*!*i> 

_u=  1  i=  1 

_l=l  U=1 


j=l,2 . m. 


(B.  18) 


Note  that 


E(A'A.)2  =  E  [  t  (^ui)4  +  2  E  (Ay  i)2(Ayvi)2],  i=U . r. 

l_u=l  u<v 


Since  Ay  ■  has  the  uniform  distribution  U (-6- ,  <5^ ),  i=l,2,..,r,  then  for  u— 1,2, ...,n, 


(B.19) 


E(Ayui)2  =  <r?  =  «?/3, 


i=l,2,...,r, 


E(Ayui)4  =  (9/5)0-?,  -  i=U . r. 


Furthermore,  Ayui  and  Ayvi  are  statistically  independent  for  u^v.  Formula  (B.19)  can  therefore  be 


written  as 


E(A[A;)2  =  +  n(n-l)]<rf. 


i=l,2,...,r. 


From  (B.2)  and  (B.3)  we  also  have 


v  * 

0'  <t>  ■  =  -  -=7  (v  •  -  y-)y  •  ■+  -iJi  u=l,2,...,n;  i=l,2 

-  ui  -  ui  2  4  '*  ui  1 1 ui  4 


:uirui  ~  2  4  'Jui  Ji,Jui  4' 

wr  w-  w. 

1  1  1 


S'  <j>  .0  .  =  r-  i=l,2,. ...r, 
-ui”ui  1  ’ 

U=1 


r  j  =  jjn-l)w?  +  ny2~|/w^ ,  i=l,2 . r 


Using  (B.20),  (B.22),  and  (B.23),  inequality  (B.18)  can  be  expressed  as 


E  £lj(AS),(AS)|jJ2  <  E  J  r.Aj'Aj 


E  .t^Ai'Ai^  +  ^Eri^A.lAjKA'Ag) 


=  ^  +  n(n-l)]  t  r?<rf  +  2  X>(rgE(  A^^  A'  A( 

=  [t  +  n(n-1)]i|1ri,yi +  2n^£ 


since  by  (3.3), 


E(A|Aj)  =  tr(cr2In)  =  n  <r2,  i=l,2,...,r. 


From  (B.15),  (B.16),  and  (B.24)  we  finally  get 


Var(Aj)  <  [^  +  n(n-l)].i  ■ r?<rf  +  2n2  ^  r.r£<7?<r2  .  |E(Aj)J2,  j=l,2 . m.  (B.25) 


By  substituting  the  rnean  of  Aj  from  (3.9)  in  (B.25)  we  obtain  the  upper  bound  given  in  (3.10). 
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Table  1.  Data*  for  the  Isomerization  of  a-Pinene  at  189. 5"  (Example  1) 


>1 

a-pinene 

y  2 

dioentene 

-v3 

ailo-ocimene 

-v4 

pvronene 

-  5 

dimer 

88.35 

7.3 

2.3 

.4 

1  75 

76.4 

15.6 

4.5 

7 

2.8 

65. 1 

23.1 

5.3 

1.1 

5.8 

50.4 

32.9 

6.0 

1.5 

9.3 

37.5 

42.7 

6.0 

1.9 

12.0 

25.9 

49.1 

5.9 

2.2 

17.0 

2.6  21.0 


14.0 


57  4 
83.1 


5.1 


LI 


25.7 


Table  2.  The  Matrix  BB/  and  its  Measures  of  Ill  Conditioning  (Example  1) 


■vi 

-v2 

y3 

y4 

-v5 

•V1 

1.0 

-.9997 

-.3679 

-.9996 

-.9852 

-2 

-.9997 

1.0 

.3805 

.9993 

.9817 

BB' 

y3 

-.3679 

.3805 

1.0 

.3557 

.2105 

-v4 

-.9996 

.9993 

.3557 

1.0 

.9868 

V- 
-  D 

-.9852 

.9817 

.2105 

.9868 

1.0 

Variance 

inflation 

224,115 

138,165 

370.77 

3,781.55 

20,624.1 

factors 


Condition 

number 


*(B')  =  1258.08 


Table  3. 

Values  of  Aj, 

E(Aj),  ny 

and  the  Eigenvectors  of  BE* 

(Example  1) 

j 

1 

2 

3 

4 

5 

Aj 

4.11521 

.881599 

.0027773 

.0004080 

.0000026 

E(Aj) 

.0002361 

.0004034 

.0000732 

.0006083 

.0000068 

’j 

153.018 

32.7703 

.100552 

-.007451 

-.000157 

Orthnotmal 

.492148 

-.059831 

.109986 

-.399853 

-.763041 

eigenvectors 

-.492328 

.045061 

-.491572 

.39444 

-.598629 

of  BB' 

-.206855 

-.966695 

.14714 

-.010939 

-  030677 

-.491659 

.073606 

-.253694 

-.82648 

.073646 

-.480505 

.233392 

.812557 

.036687 

-.230319 
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Table  4.  Values*  of  the  Breaking  Force  and  Sensory  Response  Variables  (Example  2) 


*C'^.BaE 

1  "  1 

1  *  J 

'*V  '  &M  |  g|  „  ;»  ■  '.,i!*?y 'fif-Yg' 

1  V  T o- 'w  '  J *ty\\  ’y-  BmuibB;':  ■  *-,'3 

r y*  o- ' Jv -  j 

-*  >  '*-■  3" « 

\  :,v .;  vv  \c.-?3--  k^EJEx^^B- 

M y ■'  ^Kli 

v.  ‘y-j,  yy; £'■ 

'£;V  *,'3 'r‘3  ’’i  *■» 

j  y  ~-Z > -C:’ 3 C' 

J 

V.‘.’^";,l ,  ’  'A"-i  .'  '^HtRuj^l  •'•  V-;/-  '^rc  ,(^7* 

1 

.’•7  1  V’. 

1  J 

W.J 

Pa 


♦Source  :  Ahmed  et  al.  (1983) 

♦  ♦The  response  values  are  based  on  a  scale  from  1  to  9. 
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